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ABSTRACT 

In this article we will discuss part of a piece of research that was 
conducted with two 4ESO groups. Textual learning is opposed to 
multimedia learning within the context of PLE's (Personal Learn¬ 
ing Environment) reading tools and strategies. In the research an 
analysis was made of whether it would be possible to improve the 
reading process through multimedia over a school term in two 
different aspects; one evolutionary with six classroom exercises 
and one evaluation with a final exercise. Concretely, this article 
states the number of question mistakes that the students made. 
The data indicates that there is a better evolution in students that 
performed the multimedia dynamic, although there are not any 
relevant differences in the final evaluation. 
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1 INTRODUCTION 

The irruption of digital tools in education has made Social 
Sciences take a particular interest in their study. Within this 
field, there is a crucial element: the democratisation of video. 
While in the past it was necessary to have more complex 
technical media (such as projectors, digital carts), today it is 
quite simple for a teacher to have the appropriate tools to show a 
video in a classroom. To this we can add the estrangement of 
teenagers from written media and their major exposure to 
multimedia. In line with this idea, it should be mentioned that 
there is a strong movement towards autonomous multimedia 
learning for teenagers, who are increasingly including it in their 
daily practices. A significant part of this movement takes place 
through a video platform called YouTube. It was conceived as a 
simple no-place where teenagers could share their videos, to 
become a real element of change, able to convene, report and 
provoke movements at a high political level; what Moises 
Naimm calls The YouTube Effect (Naimm, 2007 The flexibility 
and diversification of this platform have been studied, for 
example in high education (C. Orus et al., 2016) proving to be a 
very useful tool to create a more active and autonomous form of 
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learning from the students. In this learning context, completely 
linked to the idea of collaborative and autonomous learning of 
the PLE, is where this article is based. Subsequently, we will try 
to determine exactly which instance of the PLE we are studying. 

In 2007 Graham Attwell (Atwell, 2007) indicated that the PLE 
concept (then in its infancy) recognises a continuous and auto 
regulatory learning from the subject, in different contexts and 
situations. His reference was the discussions that took place in 
2006, in the Association of Learning Technology’s annual 
conference. Likewise, we can see that in 2008 the concept was 
still being discussed; so, van Haremelen (van Haremelen, 2008) 
was widening Atwell’s idea of the PLE by adding concepts such 
as planning, objectives and, what’s most important, that it was a 
process that could be performed autonomously or with the 
support of others. 

In both definitions, a reference was made to generic aspects of 
the term; therefore we will use 2010’s Jordi Adell’s and Linda 
Castaneda’s definition, which is more operational, and in which 
they tell us that the PLE is: “a set of tools, information sources, 
connections and activities that each person uses on a regular 
basis to learn” (Adell & Castaneda, 2010, p. 23) We are 
especially interested in how they define the parts of the PLE into 
3: 

(1) Tools and strategies for reading (understood in the 
broadest sense): the sources of information as an object 
or device (media libraries). 

(2) Tools and Strategies for reflection : environments or 
services in which information is transformed, mixed and 
re elaborated (sites where someone can write, comment, 
analyse, recreate, evaluate, recommend, publish, etc.). 

(3) Relationship tools and strategies: environments where 
we can interact with people that we learn from and 
people that we learn with (Castaneda & Adell, 2011, p. 
89). 

In this article, we are going to focus on the first part. In order 
to establish the base of a PLE it is crucial that the subject starts 
from a documental basis. This basis is traditionally a textual one. 
In our research we performed a text comprehension study by 
comparing text material versus multimedia material. Again, 
Adell and Castaneda give us a table to express the type of 
materials that may be part of the PLE’s first part. (Castaneda & 
Adell, 2013, p. 16): 
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Table 1 . Knowledge sources 


Tools: newsletters, blogs, video 
channels, RSS lists, etc. _ 

Mechanisms: searching, curiosity, 

To read/access to the information initiative, etc _ 

Activities: lecture, reading, 

headlines revision, multimedia 
watching 


Our research explores the possibility of performing two of the 
activities set forth in Table 1 as one: the textual reading and 
multimedia reading depending on a single general cognitive 
ability: human understanding. This idea is defended by Ann 
Morton Gernsbacher (1995) and she calls it: Structure Building 
Framework ; according to this theory, human understanding is 
based on the formation of mental structures. In the process, we 
can find at least three components. First, Laying a Foundation 
(Gernsbacher, 1995, p. 289), is nothing more than the 
preparation, the foundation on which the structure will be held 
by the Agent of Comprehension or “comprehender”, which uses 
the first information received to initiate this process. The second 
is the Mapping (Gernsbacher, 1995, p. 290); if the incoming 
information is coherent with what we have in the formed 
structure, it will accumulate in layers and increase or enhance it. 
Finally, the third process is Shifting (Gernsbacher, 1995, p. 291), 
whereby if the incoming information is not coherent with the 
structure, the latter will be displaced, and the construction of a 
new substructure will start. What seems more interesting about 
this process is that the adults of advanced level, with a good 
ability to understand written language, have a similar ability to 
understand spoken language. This is one of the reasons why 
Gernsbacher proclaims that many of the processes and 
mechanisms involved in language comprehension are general 
processes and mechanisms. Therefore, the reasons why adults 
differ in their comprehension skills are not specific to language 
in fact the individuals differ in their abilities to build coherent 
mental representations, regardless of mode. 1 


That is why it is best to call a marker to represent their skill in 
understanding. Said marker, (denominated Comprehension 
General Skill which was defined by Perfetti, Goldamn and 
Lesgold) refers to poor access to information recently 
assimilated by listening to or reading texts (Perfetti and 
Goldman, 1976; Perfetti and Lesgold, 1977). Gernsbacher, 
Varner and Faust extend the meaning to visual texts as well 
(Gernsbacher, Varner and Faust, 1990, p. 432). In general, the 
comprehension agents have very little disposition to 
remembering the information they have recently understood, and 
even more so if it is superficial. In summary, if an individual is 
less qualified to read or understand an oral text, they will also be 
to understand a visual presentation, provided that it concerns 
equivalent contents 11 

Bellow, an outline of the theory of self-development can be 
seen (Figure 1). 

Along with this idea of reading as a cognitive general ability, 
we explore another theory that serves as a theoretical 
framework: The multimedia learning (Mayer, 2005, 2008, 
2011). Mostly belonging to Richard E. Mayer and based on a 
principle he called “Principle of Multimedia Learning” (Mayer, 
2005, p. 1), it tells us that people can learn in a more profound 
way if the material is presented in multimedia format rather than 
in a simple format (written words, still images...) According to 
Richard E. Mayer (Mayer, 2005, 2008, 2011), in order to learn 
multimedia material, it is necessary to process it through two 
channels: sight and hearing, the processing capacity is limited 
(Sweller, 1988, p. 261) and active. There are three types of 
memory in the process: the sensory, the work memory, and 
finally, the long-term memory where the meaning is integrated. 
It is, in short, a process rich in cognitive implications. For 
Mayer, there are three principles on which he bases his theory. 
These are the “Principle of the two channels” (sight and 
hearing), the “principle of the limited capacity” that each learner 
can support each input, and finally, the “principle of active 
processing” (Mayer, 2011, p. 82). 
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In order for the information received by the receiver to 
become knowledge, it is essential that it exercises some active 
functions: selecting relevant information, organizing it within 
relevant cognitive structures and of good quality, and finally, 
integration into prior knowledge to turn that information into 
integrated knowledge in the long-term memory. These functions 
correspond to the memories mentioned above. That is, the 
individual must be mentally aware of the process for optimum 
processing, which is metacognition. Although we will not 
develop this point, it is also associated with PLE in the reflexion 
aspect in the individual line, without making reference to the 
PLN. 

Again, an outline of the self-development theory is attached 
(Figure 2). 

2 OBJECTIVES AND HYPOTHESIS 

Based on this theoretical framework, during the school year 
2012/2013, we conducted a study on two groups in 4th grade of 
Secondary Education (ESO-15 years old students) in Agora 
Portals International School, a private school located in Mallorca 
(Spain) with high social class families with high purchasing 
power. The research problems we decided to confront were the 
following: Is there a difference between working on the reading 
comprehension of fourth ESO students with texts rather than 
working on it with multimedia? What differences can we see in 
the process? Is its behaviour different compared to a multimedia 
test? 

To try to answer these questions, a series of hypotheses and 
sub-hypotheses were established. Fundamentally, two aspects 
were studied: evolution and assessment. In this article, we 
explore two sub-hypotheses from the research. 

Sub-hypothesis 1 (HI): "It is expected to observe significant 
differences in the evolution of the number of question mistakes 
that a multimedia and a textual learner makes during classroom 
exercises. Specifically, it is expected that the number of question 
mistakes in multimedia learners will decrease more sharply than 
in the case of textual learners”. Going back to the introduction, if 
we remember what Gemsbacher tells us, reading comprehension 
is not a specific ability but a general ability, which covers a 
broader range of cognitive processes. Among them would be the 
multimedia comprehension. Following this reasoning, by 
working on multimedia reading comprehension we are helping 


its textual improvement. This reasoning does not help change the 
forming methodology if a textual field is simply replaced with 
and multimedia one. This is why it is pertinent to add Mayer’s 
theory here. As explained earlier, multimedia methodology can 
help a student’s motivation towards knowledge, and it is a more 
complex cognitive understanding process than textual 
comprehension. However, it should be noted that by being more 
complex does not mean that it has to be beneficial per se, but, 
with a good design, it is a methodology that greatly helps their 
pedagogical effectiveness (Fletcher and Tobias, 2005, p. 120). 
Taking in consideration these two theories, we have the 
theoretical explanation of the hypothesis in question. When the 
teacher tries to get its pedagogical objectives, a multimedia 
methodology can be a very useful tool, better than a traditional 
one, which is very far away from the multimedia environment in 
which the students interact. 

Along with these concepts, this factor was also studied in a 
timely assessment with sub-hypothesis 2 (H2): “We expect to 
observe significant differences in the number of question 
mistakes that the multimedia and textual student makes during 
the final evaluation test. Specifically, we expect to find fewer 
mistakes in the multimedia students”. 

3 MATERIALS AND METHODOLOGY 

The research was carried out for one year with two groups of 25 
students each. 

For textual support, Language and Literature 4°ESO, the 
Editorial Oxford, Adarve-Trama series was used. Multimedia 
materials were also sourced on the web and exercises were 
created responding to the four areas proposed by the textbook to 
work on reading comprehension: 

(4) Extraction of inferential information. 

(5) Extraction of literal information. 

(6) Lexicon. 

(7) Spelling and grammar. 

Thus, the work of six teaching units on reading 
comprehension in the textbook was chosen specifically. In 
addition, the visual exercises were removed, leaving only the 
purely textual that responded to the four areas previously 
mentioned. 



Figure 2. Multimedia Learning graphic summary 
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Regarding the multimedia exercises, the classroom blog was 
used as support, because it offered many possibilities, it was 
simple, and the students were accustomed to working with it 
since it is often used. 

The selection of the sample of participants was not carried out 
probabilistically, indeed, it was a non-probability convenience 
sampling, where participants have been deliberately chosen as 
research participants because they are more suitable than others 
to meet the objectives of analysis. It couldn’t have been done 
differently since the researcher was required to collect data 
while performing his daily teaching in the context explained 
above. 

Although it was controlled that the answer would not be 
affected by certain confounding variables (ensure a balanced 
level of reading), the fact that the chosen students have a defined 
profile (determined by the educational context of a high social 
and economic level private school) makes the generalization to 
other educational contexts (public schools or schools with 
another student profile) impossible. 

4 FIELDWORK 

Firstly we effected a reading evaluation of the two groups to 
discover if the level was even. To this end, the test PROLEC 111 
was used, prior to family authorization and in collaboration with 
the Student Support Department. 

Once it was found that the level was similar and that there 
were no participants who scored well below the normality, the 
field research started. Six exercises of each type were 
performed. In the exercises, we worked on the four areas 
previously mentioned. 

Both the textual and multimedia exercises worked with different 
textual fonnats and multimedia materials. 

The multimedia exercises that the students carried out were 
being uploaded to the classroom’s blog, 
http://jugarconlaliteratura.blogspot.com/, with the tag 
“Comprension lectora this way, having videos in the blog and 
the text of the questions that were being used, the students could 
refer to them as the same way as the students that were doing the 
exercises with their textbook at home. 

The classroom was equipped with a digital cart, a portable 
computer per student, and internet. The teacher brought 
headphones from home so they could watch the videos (all 
hosted on the digital platform YouTube), as many times as they 
wanted. The two study groups carried out the exercises on paper 
and by hand. 

The correction was made collectively in the classroom, and 
then again by the teacher. Both, the study group of students who 
did multimedia exercises, hereinafter “MULT”; as well as the 
study group carrying out textual exercises, hereinafter “TXT” 
could refer to their text or video as many times as needed. 

Monitoring student’s development was done individually and 
using the following table: 

Table 2. Monitoring chart 
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The student, as a rule, needed two sessions to perfonn the 
exercises and correct them, both multimedia and textual. 

After completing all the exercises, at the end of the course, the 
teacher prepared an assessment test. It was intended to be 
perfonned in one session of forty minutes, and the evaluation 
criterion was the same as in the exercises. The only exception 
was that both the MULT and the TXT were only allowed to 
watch the video once. 

5 DATA ANALYSIS 

The performed statistical analyses have allowed putting the 
hypothesis of this study to the test. 

To this end, we used the technique of analysis of variance 
(ANOVA) with repeated measures to study the evolution in 
reading comprehension depending on the methodology used 
(multimedia or textual) (HI). For H2 (the relationship between 
performance in reading comprehension and methodology) the 
technique of comparison of two independent samples was used, 
in its parametric side (t-Test) when it meets the assumption of 
normality in each group or its non-parametric side (Mann- 
Whitney U test) when such normality does not occur. 

Also, effect size measures, as Eta-squared (rj 2 ) (for ANOVA), 
Cohen’s d (for t-Test) are provided, in order to provide 
infonnation on the practical relevance of the relationships that 
have resulted as significant. 

6 RESULTS 

With the intention of responding to the hypotheses that we raised 
in the introduction, we will present the results in two sections. 
The first section reflects the analysis of data obtained during the 
exercises throughout the course (longitudinal analysis). The 
second presents the results in the final evaluation exercise. 

6.1 Results throughout the year 

We analyse, longitudinally, if there are differences in the 
measure of question mistakes by exercise (see summary values 
in Table 3). Specifically, we analyse whether one group has 
more question mistakes (wrong answers) than the other. 

Table 3. Questions mistakes per exercise 


Exercise 


TXT 

MULT 

Total 


AVG 

23,76 

11,8 

17,78 

1 

SD 

8,151 

4,805 

8,963 


N 

25 

25 

50 


AVG 

25,88 

9 

17,44 

2 

SD 

11,099 

4,123 

11,889 


N 

25 

25 

50 


AVG 

13,65 

14,92 

14,285 

3 

SD 

4,215 

3,04 

3,694 


N 

25 

25 

50 


AVG 

13,65 

6,15 

9,9 

4 

SD 

4,216 

1,877 

1,877 


N 

25 

25 

50 


AVG 

15,12 

17,777 

16,449 

5 

SD 

4,003 

5,302 

4,84 


N 

25 

25 

50 


AVG 

23,949 

7,16 

15,554 

6 

SD 

4,957 

3,051 

9,407 


N 

25 

25 

50 
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If we analyse the average values (Table 3, Figure 1), the 
mistakes vary throughout the six classroom exercises, differently 
in each group, with a significant interaction effect Evolution x 
Group (ANOVA analysis detailed hereafter). As can be seen 
graphically (Figure 1), the MULT group decreases its number of 
mistakes, while TXT, despite having the same number of 
mistakes in Exercise 3, then increases the number of mistakes 
again. 

The ANOVA analysis of the evolution of these mistakes (with 
Greenhouse-Geisser correction) indicates that there is significant 
interaction, F (3.3, 158.4) = 40.37,/) < .001, rp = .457, between 
the evolution of question mistakes in the exercises and the 
groups being compared. This shows that there is a different 
pattern of evolution in the decrease of question mistakes in the 
six tests when comparing the groups (TXT vs. MULT). 
Specifically, the MULT students are the ones who demonstrate a 
more positive development. 



1 2 3 4 5 6 

- -MULT - -TXT 

Figure 3. Evolution of the number of question mistakes 

Finally, if we analyse the main effect of the group, the average 
number of question mistakes (assessments averaged 1-6) is 
significantly different between the groups, F (1, 48) = 79.826; p 
< .001, the multimedia group being more accurate (M= 11.146, 
SD = 2.130) than the textual group (M — 19.346; SD = 4.064), 
with an effect size (effect of the activity on the recorded 
response) of 62% (rj 2 = .624). Therefore, MULT students make 
less mistakes than TXT students, the practical relevance of the 
differences being found high, given that the type of performed 
activity would explain 62% of the observed variability. 

6.2 Results obtained in the evaluation test 

In this section, we present the descriptive statistics of the 
measurement of question mistakes in the final evaluation test 
(Table 4). 


Table 4. Questions mistakes in the evaluation test 


Group 

AVG 

SD 

N 

TXT 

2,28 

1,173 

25 

MULT 

1,88 

1,13 

25 


To determine whether there are differences between groups, 
we used the Mann-Whitney U test (in absence of normality of 
the response variable in the groups), the degree of significance 
being p = .259. Therefore, despite an increased occurrence of 
errors in the TXT group being observed, there are no significant 
differences between the group means. 


7 DISCUSSION 

As in the Results section, the data will be discussed through two 
subsections. 

7.1 Discussion of the results obtained in the 
longitudinal measurements 

Referencing the data we have analysed regarding the question 
mistakes, let’s remember that the MULT group has fewer 
mistakes than the TXT group (p < .001). The first averaging 
11.15 (SD = 2.13) and the second 19.35 (SD = 4.1) 

As for evolution, it has been stressed that MULT students 
reduce their mistakes in a greater proportion than the TXT 
(Figure 1). With this data, we developed our second principle: A 
multimedia methodology to work on reading comprehension, 
properly designed, helps students to focus, reducing the number 
of response mistakes they make in the exercises and improving 
their evolution throughout the teaching process. 

We will discuss these results through the “Multimedia 
Principle” by R. Mayer, which confirms that through multimedia 
materials, humans leam more than with traditional materials 1V . 
Evidently, we are not discussing learning in itself here, but about 
the amount of information that a student can retain thus saving 
readings and making less answering mistakes. We are in a 
mnemonic, greatly researched by Mayer. In 2001 (Mayer, 2001), 
for example, demonstrated that using static diagrams and text 
showed a retention average of 0.86 and 1.36 of transference in 
students. This result becomes more pronounced when 
animations are being studied, since they already include image 
and sound (Mayer and Anderson, 1991; Moreno and Mayer, 
2002). This data is consistent with our results, to greater 
retention, less need to resort to the source to find information in 
order to solve the exercises. Before we continue, we must 
remember that there are researchers that are very critical of this 
principle. Clark and Feldon (Clark and Feldon, 2005, 2014) state 
that there is not enough well-designed research to assert that 
multimedia learning can achieve something that cannot be done 
through non-multimedia. They say that the experiments that are 
proving it (Mayer, 2001) are poorly designed. In fact, they 
challenge any researcher to refute them. 

In short, we see that the MULT students, contrary to what 
researchers Clark and Feldon think, need less readings to 
perfonn their activities, and besides, they make less mistakes. To 
this, we must add that their evolution is significantly better. 
They retain more information and their learning is more accurate 
by the application of the Multimedia Principle, or because the 
multimedia material motivates them more, a feature also 
allocated by Mayer to this type of educational material, and 
makes them focus more on the exercises. 

The latter is an assertion that must be done with caution, even 
within the Multimedia Learning theory there is a principle called 
“Principle of divided attention v (Ayres and Sweller, 2005). This 
articulates that you have to be cautious with multimedia design 
because it can produce an understanding that is not integrated 
into the Mayer model, having the opposite effect and dividing. 
This causes an increase in the cognitive load with the consequent 
danger of overload. Fundamentally, this process occurs when 
different built-in formats are necessary to understand the 
material that is presented at the same time, producing a negative 
effect on the student that rejects the material. This principle 
should be applied when researchers create the materials. In our 
case, although we created the questions to make the exercises as 
similar as possible to the textual, the multimedia was not, as it 
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was created by professionals (available in internet) to try and 
avoid this problem. 

7.2 Discussion on the results of the final evaluation 
test 

Now we proceed to discussing the results obtained in the 
evaluation test vl . We have already noted that the comparison of 
the averages of the final exam results, regarding the question 
mistakes, did not indicate the existence of a significant 
difference between the mistakes of one or the other group (Table 
4 ,p = .259). Although the MULT have less errors, the difference 
is not significant. This result is inconsistent with the Multimedia 
Principle outlined before; let’s develop this idea. 

Learning with video has some obstacles when transferring 
information. Since the videos are very rich in information, the 
attention of viewers can go to the accessory and disregard the 
important information (Goldman-Segall, 1998; Miller 2011; 
Roschelle, 2000); we have also explained that even this format, 
without a proper design, can lead to cognitive overload that 
makes it hard to capture the really relevant elements. In addition, 
viewers can incorporate deep convictions to small video 
fragments, with a greater intensity than to text fragments 
(Ambaday & Rosenthal, 1993; Miller & Zhou, 2007). Some of 
these problems may have contributed to no significant difference 
between the two groups, or just the opposite, they have shown 
fewer errors because the format made them more focused on the 
task. Since we don’t have data of a test with the opposite fonnat, 
we can not assert it, but it should be noted that the multimedia 
format has proven effective even in the teacher training (Derry, 
Sherin, & Sherin, 2014) field in which several studies have been 
conducted, demonstrating the efficacy (Derry, Hmlo-Silver, 
Nagarajan, Chernobilsky and Beitzel, 2006; Santagata, 
Gallimore, and Stigler, 2005; Sherin, Jacobs and Philipp, 2011) 

8 CONCLUSIONS 

The conclusions we have reached after the analysis of our data 
and the corresponding discussions are: 

A Multimedia methodology to develop the textual 
comprehension, properly designed, helps the educational process 
to be more effective because students make fewer mistakes 
when answering questions. 

Furthermore, we have seen that at the time of the evaluation, 
the introduction of a multimedia methodology compared to a 
textual presentation does not improve (at least in absolute 
terms), the student’s ability of success when responding. 

As mentioned above, the main limitation of this study is that it 
is not possible to generalize the results of other educational 
contexts, due to the use of a non-probability sampling; therefore, 
we must be cautious when applying these conclusions, only to 
the context of this research and its participants. 

As indicated in the introduction of the article, the first part of 
a PLE should combine textual reading with multimedia, given 
that they respond to common mechanisms. We have shown that 
they enrich the evolutionary process by making learning more 
accurate, although not so much in a timely assessment where we 
couldn’t see any differences in the ability of success when 
answering questions. This data, far from being trivial, agrees 
faithfully with the philosophy of PLE’s theory that seeks to 
improve the learning process and is not focused on the 
evaluation, but on the evolution. 


Finally, our investigation continues and we are expanding it to 
university contexts, with a higher study population and 
probability sampling. 
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NOTES 
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